Skip to main content

Reply to this post | See parent post | Go Back
View Post [edit]

Poster: Nemo_bis Date: Nov 3, 2013 3:22am
Forum: faqs Subject: Re: robots.txt problem

It's now User-agent: ia_archiver according to https://archive.org/about/exclude.php